@ddaspit reviewed 14 files and all commit messages, and made 2 comments.
Reviewable status: all files reviewed, 2 unresolved discussions (waiting on Enkidu93 and pmachapman).

pyproject.toml line 40 at r1 (raw file):

  "Topic :: Text Processing :: Linguistic",
]
requires-python = ">=3.11,<3.15"

Do we have to remove support for 3.10 to get something to work? It is still officially supported until October.

tests/tokenization/sentencepiece/test_sentence_piece_tokenizer.py line 24 at r1 (raw file):

    tokenizer = SentencePieceTokenizer(model_filename)
    tokens = list(tokenizer.tokenize("Other travelling salesmen live a life of luxury."))
    assert len(tokens) == 35

Why did this change?